System and method for processing transaction data

ABSTRACT

Computer system provided with a software architecture for processing transaction data which comprises a platform ( 5 ) with at least one logical processing unit ( 21 ) comprising the following components: a plurality of gates (G(k)); one or more message queues (MQ(k)), these being memories for temporary storage of data; one or more databases (DB); a hierarchical structure of managers in the form of software modules for the control of the gates (G(j)), the messages queues (MQ(k)), the one or more databases, the at least one logical processing unit ( 21 ) and the platform ( 5 ), wherein the gates are defined as software modules with the task of communicating with corresponding business components (BC(j)) located outside the platform ( 5 ), which are defined as software modules for carrying out a predetermined transformation on a received set of data.

SCOPE OF THE INVENTION

[0001] The invention relates to a data processing system for processingtransaction data.

PRIOR ART

[0002] Such a system is, for example, nowadays frequently applied inautomatic invoicing processes in the telecommunications industry. Otherexamples relate to the processing of data and orders at banks, as wellas point-of-sale records data processing. The scope of application ofthe invention is not, however, limited to these examples.

[0003] It is known that applications for processing transaction datahave the following characteristics, amongst others:

[0004] 1) Usually, very large volumes of data are processed, up toseveral hundred million per day;

[0005] 2) the processing sequence consists of: data entry (usuallyautomated), followed by a number of processing steps (controlled byreference information such as customer and product data in the exampleof invoicing), output of the desired results such as (an electronicrepresentation of) invoices as well as process and applicationinformation for auditing;

[0006] 3) comprehensive provisions are in place to protect customers'input data or restore these data in the event of incidents such ashardware or software faults, operator errors, etc.;

[0007] 4) the operational management (responding to incidents such ashardware and software faults, operator errors etc.) requires greatvigilance in order to achieve the required reliability;

[0008] 5) there is a high degree of customised solutions, i.e.developments for or extensive modifications to the specific application;

[0009] 6) such applications comprise, in addition to the actual“business functionality” (for example a description of how to determinethe price of a telephone call), a considerable degree of functionalityin order to steer the processing process in the right direction. Thiscan extend from organising the processing into a number of sub-functionsto checking access privileges and determining when transactions must becreated or closed;

[0010] 7) because of the stringent requirements, such applications areoften run on mainframe computers for the sake of reliability and therequired performance.

[0011] Such applications consist of various sub-processes and inpractice make use of shared data. This means that the sub-processes areorganised on the basis of a common state and are highly dependent on oneanother. If one of the sub-processes makes an error that is linked tothe shared data, this will affect all the other sub-processes. Partlyfor this reason, the known applications have, more in particular, thefollowing drawbacks:

[0012] (a) processing of transactions is usually done in batches;therefore, a lot of time can pass between initiation of a transactionand processing thereof, i.e., until a batch is full enough withtransactions to be processed or a certain predetermined time has passed;

[0013] (b) when only a single error occurs in the processing of a batchthe entire batch with possibly a number of transactions must be carriedout again;

[0014] (c) they are expensive, both to purchase and to run;

[0015] (d) the applications are difficult to change, which leads to along time-to-market for new functionality;

[0016] (e) the applications require considerable effort for operationaland functional management; and

[0017] (f) a number of generic problems (“non-business functionality”)associated with computer systems on which the transaction process isrunning need to be solved for each new system, which in addition tounnecessary costs also increases the chance of unnecessary errors.

SUMMARY OF THE INVENTION

[0018] It is therefore an object of the present invention to provide analternative transaction processing system that performs a transactionprocess on transaction input data streams on an individual basis in alimited time.

[0019] To that end, the invention provides a transaction processingsystem comprising:

[0020] computer means;

[0021] one or more input interfaces for providing an input transactiondata stream to the transaction processing system;

[0022] one or more output interfaces for outputting an outputtransaction data stream;

[0023] one or more memory means for storage of intermediate transactionprocessing data; characterized in that the computer means comprise oneor more series of transaction data processing means that are arranged toprocess each input transaction data stream on a single stream basis.

[0024] By providing such a series of transaction data processing meansthat operate on a single basis, no batch process is required. Thisreduces the time required to start processing input transaction data ofeach stream.

[0025] In an embodiment, the transaction processing system as definedabove comprises a platform with at least one logical processing unit andcomprising:

[0026] the one or more series of transaction data processing units, eachcomprising a gate, there being at least an entry gate, an exit gate andzero or more intermediate gates;

[0027] the one or more memory units, each comprising either a messagequeue, these being memories for temporary storage of data on afirst-in-first-out basis, or a buffer, these being memories fortemporary storage on a random access basis. In a further embodiment thegates are defined as software modules with at least the following tasks:

[0028] starting up a corresponding business component located outsidethe platform, which is defined as a software module for carrying out apredetermined transformation on a received set of data;

[0029] sending the set of data to the corresponding business component;

[0030] receiving a transformed set of data from the correspondingbusiness component, that was created by the transformation of the set ofdata by the corresponding business component;

[0031] the entry gate and each of the intermediate gates being designedto retrieve the data to be transformed from one or more of the memoryunits and to store the received transformed set of data in one or moreof the memory units.

[0032] Preferably, the transaction processing system comprises ahierarchical structure of managers, in the form of software modules.

[0033] These latter software modules are designed to control the gates,the message queues, the buffers, the at least one logical processingunit and the platform.

[0034] Such an architecture provides a very flexible system. Autonomouscomponents are defined from low to high within the system. This makes itsimple to replace components or add new components. Consequently,applications can be changed more easily and are also easier to manage.Moreover, changes can be implemented at lower cost.

[0035] The invention also relates to a method for processing transactiondata with the aid of a transaction processing system, comprising:

[0036] computer means;

[0037] one or more input interfaces for providing an input transactiondata stream to the transaction processing system;

[0038] one or more output interfaces for outputting an outputtransaction data stream;

[0039] one or more memory units for storage of intermediate transactionprocessing data; wherein the computer means comprise one or more seriesof transaction data processing units and the method including processingeach input transaction data stream on a single stream basis.

[0040] Preferred embodiments of the present invention will now bedescribed.

BRIEF DESCRIPTION OF THE FIGURES

[0041] The invention will now be explained by reference to a number offigures that are intended solely to illustrate the invention and do notlimit its scope.

[0042]FIG. 1 shows a structure of a plant as a metaphor for theexplanation of the invention;

[0043]FIG. 2 shows a structure of a plant with sub-plants for theexplanation of the invention;

[0044]FIG. 3 shows an embodiment of the software architecture accordingto the invention;

[0045]FIG. 4 shows the hierarchy of various managers defined in theinvention, i.e. software control modules.

DESCRIPTION OF AN EMBODIMENT

[0046] In order to simplify the complex structures of the prior art,several independent sub-processes are defined. There are threeunderlying concepts:

[0047] 1. Separate data streams;

[0048] 2. Separation between business functionality and implementationfunctionality;

[0049] 3. Configuration concept and management concept.

[0050] These will be dealt with one by one.

[0051] 1. Separate Data Streams

[0052] The emphasis is here on the controlled processing of data in adistributed manner. This requires reliable mechanisms with clearresponsibilities for the distribution of data. Such mechanisms perform aseries of verifications on the data in order to determine whether datahave been lost or generated during the processing. For the elaborationof a sound architecture, an industrial plant has been used as ametaphor.

[0053] In reality, a plant is managed by a plant manager. The plantmanager stands at the top of the management hierarchy, that consists ofother managers and/or people and resources that are managed. A plant canbe construed as an installation that transforms raw materials into (end)products. The raw materials enter the plant via an entrance gate, whilethe (end) products leave the plant by an exit gate. Transformation ofraw materials is carried out according to a predetermined plan and cancomprise several process stages. The various transformation stages areoften carried out in different parts of the plant, which can be referredto as “workspaces”. A plant can manufacture different products from thesame or different raw materials. If the transformation process is verycomplex and different types of raw material are transformed or combinedto produce different types of products, it may be wise to subdivide theinstallation into several (sub)plants. The plant supports the logisticsand tracing of identifiable parts of raw materials, semi-manufacturedproducts or end products. The plant can be monitored from a centralcontrol room, which observes the status of all relevant components inthe plant. From this central control room, the plant manager manages allthe managers lower in hierarchy.

[0054]FIG. 1 shows an overall layout of such a plant. The total set-upis indicated by reference number 1. The set-up 1 comprises a platform 5.Within the platform 5 is situated a main plant 7 in which at least oneworkspace 13 is located. The main plant 7 has an entrance 9 and an exit11. The platform 5 is managed by a platform manager Mp, the main plant 7by a plant manager Mf, the entrance 9 by an entrance manager Mi and theexit 11 by an exit manager Mo.

[0055] Instead of a workspace 13, a plurality of workspaces can beprovided, arranged logistically one after the other and linked by meansof conveyor belts.

[0056] The plant manager Mf controls the whole plant 7. His tasksinclude:

[0057] supervision of all subordinate managers;

[0058] instruction of subordinate managers, for example with commandssuch as “stop”, “pause”, “resume”, etc.;

[0059] taking decisions in case of problems;

[0060] receiving status information from subordinates;

[0061] provision of status information to superiors, in the situation asdrawn in FIG. 1 corresponding with the platform manager Mp.

[0062] In some cases it may be useful to distinguish between the varioustypes of sub-plants. Two types can be defined: a process plant and aproject plant. A process plant is defined for an indefinite period forthe processing of a special type of data. There may be, for example, aprocess plant for ATM invoicing (ATM=asynchronous transfer mode), whileanother process plant handles, for example, ISDN invoicing(ISDN=integrated services digital network). A project plant, on theother hand, is defined for a specific period, for example 12 months. Theclosure of a project plant is usually initiated by an event outside thesystem, which may possibly be present in the main data stream enteringthe plant. The actual closure of the project plant is the responsibilityof the plant manager Mf and/or the hierarchy of plant managers, if thereare several.

[0063] Lower in the hierarchy, a so-called non-persistent plant, forexample, can be defined. In such a plant, data are not held inpersistent memory. In this case, the data are stored on entering theplant. A restart point is defined with these data, and the data at theplant exit will be delayed until the restart point has reached the endof the plant. The entry data are then deleted. The advantage of this isthat the data can be transported more rapidly. This type of plant doesnot, however, permit the use of databases that exist longer than thelife-time of the plant.

[0064]FIG. 2 shows an example of a plant that comprises severalsub-plants. In FIG. 2 the same reference numbers refer to the samecomponents as in FIG. 1. Instead of a workspace 13, FIG. 2 has asplitter workspace 13′. This splitter workspace 13′ can be seen as aspace in which input data, which entered via the entrance 9, are splitinto parts which are operated on by various sub-plants. FIG. 2 shows twosub-plants 15(1), 15(2). Of course, there can be more than twosub-plants.

[0065] Each of the sub-plants 15(1), 15(2) has its own sub-plant managerMsf(1), Msf(2) who manages the respective sub-plant. Each sub-plant15(1), 15(2) comprises its own workspace 17(1), 17(2).

[0066] Instead of one workspace 17(1), 17(2) per sub-plant 15(1), 15(2),a plurality of sub-workspaces can be provided, linked by means ofconveyor belts.

[0067] The output of each sub-plant leaves the main plant 7 via the exit11. The sub-plants could, for example, be for processing data fromdifferent days of the month. In FIG. 2 it is indicated that sub-plant15(1) is for processing the data from day 6, while sub-plant 15(2)processes the incoming data for day 7. The process carried out in eachsub-plant could, for example relate to the valuing of call data, whichmay be dependent on the day of the month or the day of the week.

[0068] Within the schematic shown in FIG. 2, it is for example alsopossible to open an extra sub-plant for a particular day on which asub-plant is already open if it turns out that the volume of data to beprocessed is too large for the sub-plant already open. This iscontrolled by the plant manager Mf. Each of these open sub-plants hasits own data on which operations are carried out or from which new dataare derived. Different sub-plants do not make use of the same data, sothat each of the sub-plants operates in its own state.

[0069] Within the schematic of FIG. 2 it is also possible to openfurther sub-plants and to assign to each open further sub-plant a newstream of subjects to be handled, which is split off from the mainentrance stream.

[0070] In the schematics shown in FIGS. 1 and 2, the main andsub-responsibilities for the platform manager Mp, the plant manager Mf,and the sub-plant managers Msf are clearly defined by means of ahierarchical structure. The structure shown in FIGS. 1 and 2 can betranslated into a functional design for the envisaged system, as shownin FIG. 3.

[0071] 2. Separation Between Business Functionality and ImplementationFunctionaly

[0072]FIG. 3 shows a possible implementation of the platform 5 and someexternal applications 3. The platform 5 provides the implementationfunctionality, while the external applications 3 constitute the businessfunctionality. Together they are able to process individual data streamson a stream-by-stream basis in a very efficient and reliable way. Anexample of such functionality is CDR (Call Detail Record) processing.Then, data related to individual payment transactions, e.g., debitingfrom an account is collected and processed to present it in apredetermined format to an account holder.

[0073] The platform 5 comprises a plurality of gates G(j), j=1, 2, . . ., J, of which 5 are shown by way of example. The gates G(j), togetherwith the applications 3, are comparable with one or more workspaces fromFIGS. 1 and 2. Furthermore, the internal platform 5 comprises aplurality of message queues MQ(k), k=1, 2, . . . , K, of which 3 areshown by way of example, a database DB and platform management processes19. Of course, several databases can be provided, if necessary. Theplatform management processes 19 are shown as one block. These processescan be implemented on one processor, on several parallel processors oron a network of processors. If there are several computers, they areconnected to one another via a network (for example an intranet).

[0074] The external applications 3 include business components BC(j)(one business component BC(j) corresponds with each gate G(j)), entrycomponents I(m), m=1, 2, . . . , M, of which 2 are shown, and outputcomponents O(n)=1, 2, . . . , N, of which likewise 2 are shown, and aclock C.

[0075] Each component within the platform 5 has its own manager, i.e.,is controlled by a program part designed specially for this purpose inthe platform management processes 19. This is indicated by the dottedlines between the platform management processes 19 and the othercomponents in the platform 5. The platform 5 itself also comprises amanager—the platform manager—and only this manager has no manager higherin the hierarchy. What is shown in FIG. 3 can be conceived as a platformwith one “plant” with several “workplaces” (G(j)) which are linked toone another by “conveyor belts” MQ(k). There is a strict hierarchy: thehighest in the hierarchy is the platform manager, who manages all plantmanagers, who in turn manage gate managers, message queue managers anddatabase managers. These managers all form part of the platformmanagement processes 19. Status reports are kept for each component andsent “up” via the hierarchical lines so that managers are aware of thecondition of that part of the system for which they are responsible.

[0076] Such a management hierarchy enables the different components ofthe architecture to be distinguished from one another and managedseparately. Such a hierarchical architecture is easier to upscale ordownscale than an architecture with only one manager. Moreover, thereliability is enhanced because failure, for example, can take place atthe level of the various “plants” and does not need to happen at thelevel of the entire system.

[0077] Each business component BC(j) is started by a corresponding gateG(j) and represents a particular business functionality, for example theacceptance of a call record of a telephone call, the calculation of theprice and the refunding of the calculated price to the correspondinggate G(j). Starting can, e.g., be done in a runtime environment, e.g.,by means of MS Com+™ or Enterprise Javabeans™. Another example: abusiness component BC(j) could accept all available call records fromone customer, determine the appropriate bill and report the bill in theform of a data structure to the corresponding gate G(j). Businesscomponents BC(j) do not form part of the internal platform 5 and can beacquired by external software developers. They can, as it were, be“plugged” into the platform 5, on condition that they are provided witha suitable interface to enable them to communicate with a correspondinggate G(j). The business components BC(j) are themselves “stateless” andnon-transactional. The business components BC(j) perform transformationsby which data can be changed. All components that can perform aparticular process step on particular data and therefore can change astate of the data are located within the platform 5. If necessary, abusiness component can participate in a transaction which takes placewithin the platform 5. Conceptually, however, there is a strictdistinction: transactions within and transformations outside theplatform 5. That is why the changes are only effectuated under thecontrol of the platform 5.

[0078] Monitoring of the integrity of the management hierarchy, i.e.,whether or not the gates G(j) are still functioning well is done by theplatform management process 19. They are also capable of restoringfaulty gates G(j).

[0079] There are various types of gates:

[0080] A plant entry gate G(1); this is the gate belonging to the firstbusiness component BC(1);

[0081] A plant exit gate G(5); this is the gate belonging to the lastbusiness component G(5);

[0082] A platform gate G(1); this is the gate belonging to the firstbusiness component BC(1) in the main plant;

[0083] A base gate G(j); this is the gate belonging to each randombusiness component BC(j);

[0084] It should be noted that each gate could combine several types ofgates. The gate G(1) that for example belongs to the first businesscomponent BC(1) is a platform gate as well as a plant entry gate andalso a base gate. The correct functions of the gate will be activatedaccording to the type of gate.

[0085] Each gate provides a gate business component interface. In anembodiment the business component BC(j) can access a particular subsetof data by means of one or more key values (“keys” in databaseterminology). This makes it possible, in the case of intermediatestorage in a database, where ordering of data can be relevant, tospecify a preference for particular data, for example particulars of aspecific customer.

[0086] The data within the platform are usually held in the messagequeue MQ(k). Instead of this, however, it is possible for data to bestored in the database DB. This is usually done if it is wished to storedata for a lengthy period and it is wished to be able to selectsubgroups of these data, for example in order to select data for aparticular customer. In this case, the gate supports a connection withthe database DB instead of storage in a message queue MQ(k). Thisapplies for example for gate G(3). Thus, database DB functions as abuffer an can be made with standard database technology.

[0087] The functions of each base gate G(j) are as follows:

[0088] the gate starts the corresponding business component BC(j);

[0089] the gate handles process steps on behalf of the correspondingbusiness component BC(j). If something happens with the correspondingbusiness component BC(j) during the processing, the gate will interruptthe process step. Moreover, the gate G(j) will undo the process step,start a new BC(J) to perform the same process step. If this resultsagain in an error, the gate manager will report this to the managerhigher in hierarchy.

[0090] The gate will determine a weighting factor for the entry data ina process step and redistribute it over the exit data from this processstep. If data have a weighting factor “0”, this will be interpreted asan error.

[0091] The gate will report to its own manager, the so-called gatemanager, the number of generated or reduced data units. The gate willdetermine this number by calculating the difference between the numberof entry data units sent to the business component BC(j) and the numberof exit data units received from the business component BC(j).

[0092] The above-mentioned weighting factor per entry data unit will beassigned by the platform gate.

[0093] The plant entry gate G(1) will report to its gate manager thetotal number of entry data units per transaction and the total weightingfactor for these entry data units. The plant exit gate G(5) will reportto its gate manager the total number of exit data units per transactionas well as the total weighting factor for these exit data units. Thetotal weighting factor for the entry data units compared with the totalweighting factor for the exit data units allows for platform evaluation.A detailed explanation of using weighting factor is given in section 4.

[0094] The function of the message queues MQ(k) is to hold data in asafe manner, as well as to transport data between the successive gatesG(j).

[0095] It can be seen that the internal platform 5 provides forimplementation functionality, such as:

[0096] system configuration and installation;

[0097] operational management and the necessary user interfaces;

[0098] reliable data transport;

[0099] evaluation and accounting;

[0100] transactional processing of data (ACID=atomicity, consistency,isolation, durability);

[0101] load distribution; e.g., when too much data is arriving forexisting business components BC(j) to handle, the platform managementprocesses can initiate a further business component BC(j) to assist;

[0102] calling and control of the business components BC(j).

[0103] Each business component BC(j) can be designed, built and executedas a straight-forward transformation of offered data and can confineitself to carrying out the transformation and returning of processingresults. The transformation is specified in terms of businessrequirements and functional requirements of the system to be built. Theother, more system oriented and implementation oriented aspects fallwithin the internal platform 5.

[0104] A functional explanation will now be given of the steps that canbe taken within the example of the architecture shown in FIG. 3.

[0105] 1. Transaction data to be processed arrive on the platform 5 inthe form of files I(1), I(2). Alternatively, data to be processes isstored in a first, input message queue MQ(ø). This can be done byreference, in the form of a file with a name referring to the actualfile.

[0106] 2. Gate G(1) periodically starts a transaction and then callsbusiness component BC(1).

[0107] 3. Business component BC(1) detects the file to be processed andthen opens the file. The business component BC(1) detects the file byits name, whether or not this file is initially stored in message queueMQ(ø) or in a separate data store outside the platform 5.

[0108] 4. Business component BC(1) reads the file name of this file andthe corresponding location and sends the file to gate G(1).

[0109] 5. Gate G(1) stores the data from the file in message queueMQ(1).

[0110] 6. If gate G(1) was able to store the relevant data, this isexecuted by gate G(1) and the process step of G(i) is closed.

[0111] The business functionality applied in steps 1 to 6 by thebusiness component BC(1) relates to the location, extension, and namingconvention of the files to be received.

[0112] 7. Gate G(2) detects a message in message queue MQ(1), starts anext process step and reads the message.

[0113] 8. Gate G(2) calls business component BC(2) with this message,the name and the location of a file to be processed.

[0114] 9. Business component BC(2) opens the file, reads thecorresponding transaction data and sends them to gate G(2).

[0115] 10. Gate G(2) stores the individual transaction data in messagequeue MQ(2).

[0116] 11. If gate G(2) was able to store the data, this is executed bygate G(2) and the process step of G(2) is closed.

[0117] The business functionality performed by business component BC(2)in steps 7 to 11 relates to: format and record layout of the files to bereceived.

[0118] 12. Gate G(3) detects transaction data in message queue MQ(2),starts a further process step and reads in a number of transaction data.

[0119] 13. Gate G(3) calls business component BC(3) with the read-intransaction data.

[0120] 14. Business component BC(3) carries out a predeterminedtransformation and sends the results to gate G(3).

[0121] 15. Gate G(3) stores the processing results in database DB.

[0122] 16. If gate G(3) was able to store the data, this is executed bygate G(3) and the process step of G(3) is closed.

[0123] The business functionality performed in steps 12 to 16 bybusiness component BC(3) relates to: format and field layout oftransaction data, record and transformation specification.

[0124] 17. Gate G(4) periodically starts a next process step and thencalls business component BC(4).

[0125] 18. Business component BC(4) contains a scheduler and, triggeredby clock C sends an event message to gate G(4). The clock C provides thetrigger times for the scheduler, e.g., based on a list of events.

[0126] 19. Gate G(4) stores the event message in message queue MQ(3).

[0127] 20. If gate G(4) was able to store the event message, gate G(4)does that and the process step of G(4) is closed.

[0128] The business functionality performed in steps 17 to 20 bybusiness component BC(4) comprises: scheduler functionality in which itis determined that a particular business event must take place.

[0129] 21. Gate G(5) detects the event message in message queue MQ(3),starts a next process step and reads the event message.

[0130] 22. Gate G(5) calls business component BC(5) with this eventmessage.

[0131] 23. Business component BC(5) interprets the event message.

[0132] 24. Business component BC(5) formulates a request to retrievespecific data from database DB and sends these specific data to gateG(5).

[0133] 25. Gate G(5) retrieves the requested data from database DB andsends them to business component BC(5).

[0134] 26. Business component BC(5) performs a transformation on thedata and produces output, for example in the form of a file O(1) orpaper output O(2). The output can also have any other customary form,for example an automatic e-mail. As a further alternative, an outputmessage is sent to gate G(5) which stores it in output message queueMQ(4).

[0135] 27. If gate G(5) receives a confirmation from business componentBC(5), the process step of G(5) is closed.(If gate G(5) stores theoutput message in output message queue MQ(4) it directly terminates itsprocess step).

[0136] The business functionality performed in steps 21 to 27 bybusiness component BC(5) comprises: interpretation of the event message,selection of relevant data and transformation thereof into performance.

[0137] 3. Configuration and Management Concept

[0138] The management hierarchy provides a manner to achieve operationaland functional management by means of aggregating and distributinginformation about the operation of the platform 5 via a hierarchy ofmanagement components within the platform 5. FIG. 4 shows an example ofa management hierarchy in accordance with the invention.

[0139] The platform manager Mp can send management data to aconfiguration monitor Mc. In addition, management data can be exchangedbetween the platform manager Mp and the processing unit manager Mf. Theprocessing unit manager Mf is comparable to a plant manager from FIGS. 1and 2 and controls various other managers within the “plant”, which incomputer terms could for example be a logical processing unit 21. Ifthere are several logical processing units, which for examplecommunicate with each other via a network, there are also severalprocessing unit managers Mf, namely one per logical processing unit.

[0140] The other managers include a number of gate managers Mg(j) andmessage queue managers Mq(k). The gate managers Mg(j) control the gatesG(j). The message queue managers Mq(k) control the message queues Mq(k).The other reference numbers in FIG. 4 refer to analogous components asin the previous figures.

[0141] The processing unit manager Mf, the gate managers Mg(j) and themessage queue managers Mq(k) all belong to the platform managementprocesses 19.

[0142] The various managers who form part of the platform managementprocesses 19 have the following functions.

[0143] The tasks of the platform manager Mp are as follows:

[0144] The platform manager Mp takes care of starting the supportingplatform components and the processing unit manager(s) Mf when theplatform 5 is initialised.

[0145] The platform manager Mp must respond to error messages, forexample by restarting or relocating components. Error recoveryalgorithms may be further developed than merely the switching off orrestarting of a component that has a problem. If an error cannot besolved automatically by the platform manager Mp, this is announced viathe configuration monitor Mc to the operator.

[0146] The configuration monitor Mc must send a sign of life to theplatform manager Mp. If such a message is not sent, the platform managerMp will restart the configuration monitor Mc. The platform manager Mpwill then also send instructions to all subordinate managers to enablestate reports of all components from the platform 5 to be sent to theconfiguration monitor Mc.

[0147] The processing unit manager Mf controls the entire logicalprocessing unit 21 and has as subordinates the message queue managersMq(k), gate managers Mg(j) and database managers (not shown in FIG. 4).

[0148] The processing unit manager Mf has five functions:

[0149] 1. When data arrive at the logical processing unit 21, theprocessing unit manager Mf starts the components within the logicalprocessing unit. Those parts which have not yet received data are notstarted.

[0150] 2. In most cases, logical processing units (“plants”) will not beclosed, because when no data need to be processed all gates and businesscomponents are included in a group and so most of the resources whichare claimed by a logical processing unit will be free.

[0151] In some cases, however, for example if a plant processes data fora limited period, the plant will have to be closed after some time. Theprocessing unit manager Mf will then receive a closure message. Such amessage differs from a stop message, because data may be present in astopped plant. If the processing unit manager Mf receives a closuremessage, he will want to check that the plant does not contain any data.If this is the case, the processing unit manager Mf will make allsubordinate managers stop, check whether the plant is in balance (seealso below under 5) and store all relevant information such as logs andinternal states in a safe place for reference.

[0152] 3. The processing unit manager Mf must respond to error messages,for example by restarting or relocating components. The discovery oferrors at this level is always within the context of the logicalprocessing unit 21. If problems occur outside the logical processingunit 21, these cannot be solved. Such problems can only be reported tothe platform manager Mp (or a superior processing unit manager ifpresent).

[0153] 4. The processing unit manager Mf receives reports from hismessage queue managers Mq(k). Based on these reports, the processingunit manager Mf can decide to duplicate any business components BC thatmay be present, and consequently gates and gate managers with whichbusiness components have a one-to-one relationship. This can occur ifthe processing unit manager Mf observes that one of the message queuesMQ(k) is too large. The processing unit manager Mf will additionallyhave to explicitly monitor the statistics of the gates G(j) in order todetermine whether the number of business components BC(j) can bereduced.

[0154] 5. The processing unit manager Mf receives all entry weightingfactors and exit weighting factors of the logical processing unit 21 viathe gate managers Mg(j). It is the responsibility of the processing unitmanager Mf to calculate these and, after failure of the logicalprocessing unit 21, take stock of the situation. Furthermore, theprocessing unit manager Mf can ask all message queue managers Mq(k),database managers and any sub-processing unit managers to notify him ofthe weighting factors present at that moment in the logical processingunit 21. In this way, the processing unit manager Mf can determinewhether the logical processing unit 21 is in balance, even if data arepresent within the logical processing unit 21.

[0155] Each message queue manager Mq(k) has the following threefunctions.

[0156] 1. The message queue manager Mq(k) monitors the message queueMQ(k) continuously and reports to the processing unit manager Mf in thefollowing cases:

[0157] a message queue MQ(k) receives original data, so that thecomponents situated downstream can be activated;

[0158] a particular message queue MQ(k) grows beyond a predeterminedthreshold value that was configured beforehand as part of the propertiesof the relevant message queue MQ(k). If this message queue MQ(k)subsequently falls below this threshold value, this will also bereported to the processing unit manager Mf;

[0159] a message queue MQ(k) attains its maximum size.

[0160] 2. Message queue managers Mq(k) initialise the correspondingmessage queue MQ(k). The message queue manager Mq(k) is balanced inrespect of load.

[0161] 3. On request, the message queue manager Mq(k) will determine theweighting factor of the data present in the message queue and reportthis to the processing unit manager Mf.

[0162] The gate managers Mg(j) have two functions:

[0163] 1. When data arrive, the gate manager Mg(j) will initiate a gateG(j).

[0164] 2. The gate manager Mg(j) is responsible for sending on the gatestatistics to the processing unit manager Mf.

[0165] The database manager (not shown in FIG. 4) has three functions.

[0166] 1. The database manager monitors the database DB continuously forerrors and will report these to the processing unit manager Mf.

[0167] 2. Each database manager will start his corresponding databaseDB, assuming that this has been correctly initialised. Otherwise anerror will be reported to the processing unit manager Mf. Databases arenot balanced in respect of load.

[0168] 3. On request, the database manager will determine the weightingfactor of the data present at that moment in the database DB and reportthis to the processing unit manager Mf.

[0169] An explanation will now be given of the steps followed by themanagers shown in FIG. 4.

[0170] 1. Transaction data to be processed arrive on the platform 5 inthe form of files I(1), I(2).

[0171] 2. Gate G(1) periodically starts a transaction and calls businesscomponent BC(1).

[0172] 3. Business component BC(1) detects the data to be processed andsends them to gate G(1).

[0173] 4. Gate G(1) stores these data in message queue MQ(1).

[0174] The functionality of the steps 1 to 4 relates to: the inclusionof data in the platform 5.

[0175] 5. Business component BC(2) is not available on account of aninternal error.

[0176] 6. Message queue MQ(1) exceeds a predetermined limit with regardto filling with data.

[0177] 7. The message queue manager Mq(1) detects this overrun andreports this to the plant manager Mf of the logical processing unit 21.

[0178] 8. Gate G(2) detects that business component BC(2) is notresponding and informs the gate manager Mg(2).

[0179] 9. Gate manager Mg(2) reports the error situation to theprocessing unit manager Mf of logical processing unit 21.

[0180] The functionality in steps 5 to 8 relates to: the detection of—inthis example—an incorrect situation in the platform 5.

[0181] 10. Processing unit manager Mf archives the information receivedin steps 7 and 9 and determines a necessary action in a heuristicmanner.

[0182] 11. The processing unit manager Mf reports to the platformmanager Mp that business component BC(2) is in an error situation andthat the data processing is blocked.

[0183] 12. The platform manager Mp warns the operational administratorvia the configuration monitor manager Mc.

[0184] 13. The processing unit manager Mf instructs the gate managerMg(1) and the message queue manager Mq(1) to stop the data processing.

[0185] 14. The gate manager Mg(1) instructs the gate G(1) to pause sothat no new transactions are set up.

[0186] 15. The message queue manager Mq(1) instructs the message queueMQ(1) to pause.

[0187] The functionality in steps 10 to 15 relates to: taking action inthe management hierarchy in response to the detection of an error. Itwill be clear that the above-mentioned steps 1 to 15 are only an examplegiven to illustrate the hierarchical structure of FIG. 4.

[0188] Because the operational administrator has been notified in step12 of the error situation, he can intervene and correct any errors. Hecan then restart the system.

[0189] The architecture described above can be implemented with the aidof one or more computers, which for example are linked to one another bymeans of a network, for example a LAN (local area network) or a WAN(wide area network). Use can be made hereby of a standard operatingsystem, for example Microsoft Windows 2000, a message queuing service,for example Microsoft Message Queues, a component creation andcommunication service, for example Microsoft DCOM (distributed componentobject model), a distributed transaction co-ordinator, for exampleMicrosoft MTS (Microsoft transaction server), and a database system, forexample Microsoft SQL server (SQL=structured query language). Themessage queuing service and the database service must be able toparticipate in transactions under the control of the transactionco-ordinator.

[0190] The above-described architecture is flexible in many respects.First of all, the business components BC(j) can be adapted or replacedindividually. It is also simple to add new gates G(j) and acorresponding business component BC(j) within the outlined architecture.It is equally simple to add new “plants” in the form of logicalprocessing units 21, with a similar structure as shown in FIG. 4. Thismeans that changes can be implemented at a low level without the wholesystem having to be stopped.

[0191] The proposed architecture leads to lower costs for processingtransaction data than is currently possible, with equivalent or evenbetter performance, reliability and adaptability.

[0192] 4. Using Weighting Factors

[0193] Above, the use of weighting factors for platform evaluation hasbeen discussed briefly. Below, a more detailed explanation will begiven.

[0194] In any transaction data processing system it is important toprovide an as high as possible, amount of certainty about the properfunctioning of the system. This is especially true for billing systems.

[0195] The probability of certainty depends on many factors among whichthe quality of the process of software creation and softwaremaintenance, the quality of the input data, the quality of theoperational process, guarantees in the software applications themselves,reporting mechanisms, etc. Many countermeasures are directed to avoidingerror situations. I.e., software is to be delivered with an as high aspossible quality. However, measures are known to sense error situationsand to find the errors causing the error situation. In situation wheretransaction data is processed in batches, at the end of each run of abatch, a process report will be made up in which different countings arecompared with one another. Typically, these relate to countings at theinput and the output side of the processing system, e.g., number ofrecords, number of call minutes in the incoming/outgoing records, etc.These are very important reports but due to the character of the data inthese reports they are application specific. In other words, a platformwill not automatically be able to generate them. Therefore, costs arenecessary to create the software and to maintain the software. Addingrules of code in the software introduces new chances of errors.

[0196] It is observed that, in case more checks are performed, thismight have a synergistic effect, provided these checks on the properfunctioning are mutually independent. The chance that an error remainsundetected after a number of checks equals the product of the individualchances that the error remains undetected in each individual check.Therefore, it might be very effective to introduce additional checks onproper functioning, provided they are independent. The “weight” used inthe transaction processing system explained above, are such a mechanismwhich operates independently from both application directed balancingand from mechanisms of monitoring drop out of machines and processes asprovided by the management mechanisms explained above. In other words,these “weights” are an addition to other measures and are independentthereof. Therefore, they provide better security due to the synergeticeffect.

[0197] The “weights” can be performed automatically by the platform.They can be applied in two different ways: as a means to check whetheror not data has been lost but also as an independent check whether ornot a series of transaction data processing units G(1), . . . G(5) is“empty”, i.e., is not busy processing any transaction data streams (no“work in progress” anymore). This may be important in situations inwhich one wishes to “close” periodically a transaction or if one wishesto substitute an existing series of translation data processing unitsG(1) . . . G(5) for an amended series of transaction data processingunits. In the latter case, one has to know when the current series oftransaction data processing units may be closed.

[0198] It is noted that the “weights” are not intended to recover oferrors but to signal the existence of errors to predetermined processingunits or other means available for error recovery, including manualprocedures.

[0199] The purpose of using “weights” is to determine whether or not allinput transaction data streams items entering the platform 5 are alsoleaving the platform 5 at the output side. This is also true for allside products developed during processing, aggregating, etc.Alternatively, using the mechanism of “weights” provides the possibilityof evaluation whether or not there is still work in progress in theplatform 5. The mechanism is independent of the application logic andlargely independent of the correct functioning of the managementhierarchy.

[0200] A more detailed explanation of the mechanisms is as follows.

[0201] Every transaction data stream item entering the platform 5obtains a (virtual) strictly positive weighting factor. This is done byinput gate G(1), which records a total amount of weighting factorsprovided to all input data. The total amount of weighting factors may bereset periodically through a management action by factory manager Mf.All transaction data stream items carry the associate weighting factorwith them as an additional information. this is invisible for thebusiness components BC(j) but not for the gates G(1) . . . G(5).

[0202] When a gate reads a number of transaction data stream items fromhis input, e.g., from a message queue MQ(k), offers these items to abusiness component BC(j) and receives the results from the businesscomponent BC(j), the gate will redistribute the total weight of the readtransaction data stream items over all output items. All weightingfactors must be strictly positive. Then, the results are written to theoutput and only then, the transaction will be closed. In this way, thetotal amount of weighting factors will be kept constant. When leavingthe platform 5, a total amount of weighting factors is determined. Anytime, via the management hierarchy or directly via an operator action,it can be established whether or not the total amount of weightingfactors at the input of the platform 5 equals the total amount ofweighting factors at the output of the platform 5. This information canbe compared with other information determined in another way, relatingto the amount of work in progress in the platform 5, e.g., by requestingthe message queue managers and database managers in the platform 5. Whenthis information is incorrect, there is a platform error. Either data ismissing or added between transaction processing steps made by the gatesor something is wrong with the resource managers or the resourcesmanaged by them.

[0203] The software codes to allow such a functioning of the platform 5is part of the platform 5 and needs only be developed and tested once.This is an advantage from a cost point of view. But it also offers ahigher chance of being errorless than when it is developed asapplication specific code.

[0204] There are at least two reasons why the proposed mechanism ofadding weighting factors to the transaction data stream is advantageous.

[0205] First of all, middleware, such as databases and message queues,may contain errors as well. An independent check to prevent this isalways good, especially then, when they can be developed against lowcosts.

[0206] Secondly, in the concept of the platform 5 as disclosed above,transaction data streams items are “consumed” when they are processed.Input for a business component BC(j) may and should only be used once.Standard message queues have the feature that after data has been readfrom the message queue this data is not available anymore to be readonce again. Therefore, the message queues are monitoring unintendeddouble use of data. However, when a database DB is used as a buffer,this may, in general, not be true anymore. After a gate has read datafrom a data buffer, it must explicitly remove the data from the databaseDB. Therefore, an error in the software code of a gate can result indata being read more than once. E.g., a telephone call could be chargedmore than once. The opposite might also happen, i.e., data being removedprior to being processed. Then, e.g., telephone calls will never becharged to the user. Both such situations will be detected by themechanism of using weighting factors. The weighting factors provide anindependent mechanism to check the correctness of the software codes inthe gates. Although being platform code and thus being most probablycorrect, it may contain errors which are hard to find if present.

[0207] The mechanism of using weighting factors is especially importantin systems as described above in which no central “state” is recorded asin traditional database or file based systems. The mechanism provides anindependent check on the guarantees that are theoretically offered byusing middleware, such as transaction integrity during reading from andwriting to message queues and databases, and correctness of the softwarecodes in the gates providing the access to the resources.

[0208] Although being of specific importance in systems as describedhere, it is observed that the mechanism of weighting factors can also beused in other kinds of transaction processing systems, like traditionaldatabase or file based systems processing transactions in batches.

1. Data processing system comprising: computer means (G(1 . . . 5)); oneor more input interfaces (MQ(0); I(1), I(2)) for providing an inputtransaction data stream to the data processing system; one or moreoutput interfaces (MQ(4); O(1), O(2)) for outputting an outputtransaction data stream; one or more memory units (MQ(0 . . . 4); DB)for storage of intermediate transaction processing data; characterizedin that the computer means comprise one or more series of transactiondata processing units (G(1 . . . 5)) that are arranged to continuouslyprocess available input transaction data streams.
 2. Data processingsystem according to claim 1, wherein the data processing systemcomprises a platform (5) with at least one logical processing unit (21)and comprising: said one or more series of transaction data processingunits each comprising a gate (G(1 . . . 5)), there being at least anentry gate (G(1)), an exit gate (G(5); G(3)), and zero or moreintermediate gates (G(2), G(3), G(4); G(2)); said one or more memoryunits each comprising either a message queue (MQ(k)), these beingmemories for temporary storage of data on a first-in-first-out basis, ora buffer (DB), these being memories for temporary storage on a randomaccess basis.
 3. Data processing system according to claim 2, whereinthe gates are defined as software modules with at least the followingtasks: starting up a corresponding business component (BC(j)) locatedoutside the platform (5), which is defined as a software module forcarrying out a predetermined transformation on a received set of data;sending the set of data to the corresponding business component BC(j));receiving a transformed set of data from the corresponding businesscomponent (BC(j)), that was created by the transformation of the set ofdata by the corresponding business component BC(j)); the entry gate(G(1)) and each of the intermediate gates (G(2), G(3), G(4); G(2)) beingdesigned to retrieve the data to be transformed from one or more of thememory units and to store the received transformed set of data in one ormore of the memory units.
 4. Data processing system according to any ofthe preceding claims, wherein the data processing system comprises aninput message queue (MQ(0)) as input interface.
 5. Data processingsystem according to any of the preceding claims, wherein the dataprocessing system comprises a hierarchical structure of managers, in theform of software modules.
 6. Data processing system according to claim4, wherein the data processing system comprises a hierarchical structureof managers, in the form of software modules for the control of thegates (G(1 . . . 5)), the message queues (MQ(0 . . . 4)), the buffer(DB), the at least one logical processing unit (21) and the platform. 7.Data processing system according to claim 6, wherein the hierarchicalstructure of managers comprises: for each gate a gate manager including:an entry gate manager for the control of the entry gate (G(1)); an exitgate manager for the control of the exit gate (G(5); G(3)); for eachintermediate gate an intermediate gate manager (Mg(2)) for the controlof the corresponding intermediate gate (G(2), G(3), G(4); (G(2)); foreach message queue a message queue manager (Mq(k)) for the control ofthe corresponding message queue (MQ(k)); for each database (DB) adatabase manager; for each logical processing unit (21) a plant manager(Mf) for the control of each gate manager, each message queue managerand each database manager; a platform manager (Mp) for the control ofthe platform (5).
 8. Data processing system according to claim 7,wherein the platform manager (Mp) is also set up to carry out thefollowing tasks: initialisation of platform components; dealing witherror messages within the platform (5); communicating with aconfiguration monitor manager (Mc) that is set up to communicate with anoperator of the data processing system.
 9. Data processing systemaccording to claim 7 of 8, wherein each plant manager (Mf) is set up tocarry out the following tasks: initialisation of those components withinthe logical processing unit (21) associated with the plant manager (Mf)which must support a transaction on data; closure according topredetermined rules of the logical processing unit (21) associated withthe plant manager (Mf) if the plant manager (Mf) receives an instructionto this effect; dealing with errors within the logical processing unit(21) associated with the plant manager Mf); automatic reduction orduplication of one or more intermediate gates (G(j)) if the number ofdata on which a transaction must be carried out decreases or increasesrespectively.
 10. Data processing system according to any one of claims7-9, wherein each plant manager (Mf) is also set up to determine entryweighting factors and exit weighting factors of the logical processingunit (21), and to determine whether the logical processing unit (21) isin balance in respect of load.
 11. Data processing system according toany one of claims 7-10, wherein each message queue manager (Mq(k)) isset up to carry out the following tasks: initialisation of the messagequeue (MQ(k)) assigned to each message queue manager (Mq(k)); monitoringof the message queue (MQ(k)) assigned to each message queue manager(Mq(k)).
 12. Data processing system according to any one of claims 7-11,wherein each message queue manager (Mq(k)) is set up to carry out thefollowing task: determine a weighting factor of data that are present inthe message queue (MQ(k)) assigned to each message queue manager(Mq(k)).
 13. Data processing system according to any one of claims 7-12,wherein each gate manager (Mg(j)) is set up to carry out the followingtasks: initiate a gate assigned to the gate manager (Mg(j)) if dataarrive at the assigned gate; collect gate statistics and send them on toa corresponding plant manager (Mf).
 14. Data processing system accordingto any one of claims 7-13, wherein each database manager is set up tocarry out the following tasks: start up a database assigned to thedatabase manager; monitor the database assigned to the database manager.15. Data processing system according to any one of claims 7-14, whereineach database manager is set up to carry out the following task:determine a weighting factor for data that are present at a particularmoment in the database assigned the database manager.
 16. Dataprocessing system according to any one of the preceding claims, whereinthe data processing system is implemented in the form of a plurality ofpersonal computers communicating with one another.
 17. Data processingsystem according to claim 16, wherein the personal computers are linkedto one another by means of a network, for example a LAN or WAN.
 18. Dataprocessing system according to claim 3, wherein each business component(BC(j)) is designed to perform business and functional tasks of the dataprocessing system, but cannot change any state of data present withinthe platform (5).
 19. Method for processing transaction data with theaid of a data processing system comprising: computer means (G(1 . . .5)); one or more input interfaces MQ(0); I(1), I(2)) for providing aninput transaction data stream to the data processing system; one or moreoutput interfaces (MQ(4); O(1), O(2)) for outputting an outputtransaction data stream; one or more memory units (MQ(0 . . . 4); DB)for storage of intermediate transaction processing data; characterizedin that the computer means comprise one or more series of transactiondata processing units (G(1 . . . 5)) and the method includescontinuously processing available input transaction data streams.
 20. Atransaction data processing system provided with computing means, memorymeans for storage of data and instructions, an input for receiving inputtransaction data and an output for outputting output transaction dataresulting from processing said input transaction data by said computingmeans, the computer being programmed to add an input weighting factor tosaid input transaction data, to determine an output weighting factorassociated with said output transaction data, and to compare the inputand output weighting factors with one another to establish whether ornot an error has occurred during said processing of said inputtransaction data.
 21. A method of checking correctness of a transactiondata process in a transaction data processing system provided withcomputing means, memory means for storage of data and instructions, aninput for receiving input transaction data and an output for outputtingoutput transaction data resulting from processing said input transactiondata by said computing means, the method including the steps of addingan input weighting factor to said input transaction data, determining anoutput weighting factor associated with said output transaction data,and comparing the input and output weighting factors with one another toestablish whether or not an error has occurred during said processing ofsaid input transaction data.